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1. INTRODUCTION 

In the literature several optimization strategies may be originate with (theoretically) a much better 
speed of convergence than the descent gradient methods. Maybe the foremost documented ones area unit the 
conjugate gradient and quasi-Newton strategies. For details see [1]. 

Generally, for 1 number of variables of the problem has the following from: 


min {f (x) | x € R" | (1) 


where f : R" —> R' isa continuously derivable function. 
Nonlinear conjugate gradient algorithms are based on the following iterative scheme : 


0 ER 5 tp a Ga, (2) 


where the search direction d,,, is outlined as a linear combination of the present by product 77,,, and also 


the earlier search direction d ee 


dy =- > Aya =a + Bed, (3) 
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where 2, is a parameter conjugate gradient, 77,,, denotes gradient of f(x,,,) at the point x,,,, 
Sy, =Xz4, —X, and y, =1,,; —7,. More details can be found in [2]. 


The step size @, is decided in line with the Wolfe line search states as follows : 
f(x,)-f(x,+a,d,)2=-6a,nf d, (4) 
“n(x, +a,d,) d, 20nd,” (5) 


where 0< 6 <o <1 and d, isa descent direction nd, <Q. For details see [3]. 
It is well known that if the matrix of gradient is positive definite, the most efficient search direction 


at X, is the Newton direction: 


d,,,=—-(V Gia Tee = Gren (6) 


From the secant condition that : 


(Vea) Sy = Vi (7) 


More details can be found in [4]. 
The conjugate gradient methods different depend on the calculation of parameters (3, . The idea of 


variant CG methods had been studied by many researchers for example, see (Hestenes and Stiefel [5]) and 
(Fletcher and Reeves [6]). 


HS _ hae FR _ cite 
i fp, = (8) 
yy, dy Ural 





The motivation of this paper is to combine the advantages of conjugate gradient direction d oa and 


Newton direction d ae in order to provide novel parameter with better convergence. 


2. ANEW KING OF PARAMETER CONJUGATE GRADIENT 
In this section, we derive a new parameter conjugate gradient based on the three order tensor model. 
Based on the three order tensor model, the information of the second order curvature in the following from : 


5, Gui Sy = Vise + 6(f, — feat) +3 ea +1)" 5, (9) 


For more details can be found in [7]. 
The step size @, determine by many algorithms, in exact line search the step length @, is choose 


as: 


T 
ag= wee ed ae a, (10) 
d'Gd, 


We produce the steps, that lead to a new second order curvature as below : 


2 3 
8, Opis =(f; ~ Fed * 3 MS St (11) 
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which implies that : 


Ge eat? 213i), 8 —1/27, 5; i. (12) 


T 
S, Sy 





’ T T T F 
Since y, 8; =,4,5; —1];, 5,» then from the above equation, we have: 


G,,, — fa Faw) #29 p85 4 OS ae 


T 
5, Sy 





Then Newton direction can be written as : 





sis 
a a= kk . (14) 
ee Gone +1/67,,.5; 


By combine the advantages of d ae and d a , So, the equation is hold : 


Ge Weal Nest = Mer + Bey (15) 


Now, we'll realize the parameter /3, .Equation (15) multiplied by yn , then we get : 





T 
5, Sp T T T 
25 + Bd 
Gah +1/ 29, Biles ho MrVe + Bede Ye 


T 
5,5; 


(fi — fy t1/2y; 5, +1/ 6 1Si4 








Bd; y, = | Io +a Ve (16) 


from (16) we get : 


T- 
Pepe. l 5S, Sy T 
Pde Ys Gf. — faa) +1/ 29's, +1/ 67,5, ) 





then we have : 


5,8 Mes 
B, = c k“k k+l Jk (17) 
(f 





k = fry )tll2yz 5 +1/ 6154454 divs 


Then the new conjugate gradient directions are : 


58 Je 
d= - S41 k*k balk (18) 
‘i z | (fe fa t 2955, +16 Se) Oe 





For simplicity,we call equation (17) by 8/4" methods. Also 8/4” can be write in the manner : 
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I?) 
1 y 
BAH k 
kK pl] Me OT TS | es (19) 
Ve k Xk 
Where: 
rT. \2[ or T 
_ OS Ved | SNe * 5, Sx 





2 r T T (20) 
ly, | Se8p 0 (Se ~ Sea) +2738, — 1/27, 5, 


Now we are ready to state the steps of the new conjugate gradient methods. 
New Algorithms (BAH Algorithms) : 


Step 1. Give x, ER". Set k=l and d, =—7,. 

Step 2. Stop if lr, ! <10 °°. Otherwise, continue. 

Step 3. Find @,,, > O fulfilling the Wolfe states (4) and (5). 
Step 4. Set x,,, =x, +a@,d,.If leis | < 10°, then stop. 
Step 5. Compute 2, by the formulaes (19) and d,,, by (3). 
Step 6. Put k =k +1. Go to step 2. 


3. CONVERGENT ANALYSIS 
Global convergence of the BAH-algorithm will be proved in this section under the following 
assumption. 


Assumptions 
i- f(x) is bounded below on R”. ii- The gradient 7(x) is Lipschitz continuous, namely, there 
exists L. > such that : 








pon) — (x; )| S Les —Xx|p VX» % EU (21) 


Under these assumptions on f, then a constant [> O exists, such that : 


al pele (22) 


for all x € L. More details can be found in [8]. 
The sufficient descent condition has a very important property. 


3.1. Sufficient descent condition 
For the sufficient states to hold, then : 


ten S—€ Weal» ¢>0 (23) 


Theorem 3.1 
Let 5,, Yep > Mey € R",fB, ER and PB, defined by (19) where te (1/4 ,0). If 


1 
S19, 40, te dr [1 al 
r 
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Proof : 
Since dy) =—7), we have nid = Ir,” , which satisfy (23) . Multiplying (16) by 7,,,, we 
have : 
2 
T 
T 2 MeV ly, | T T 
Dis Meat =-la | 3 se PO et Se Mit (24) 
SMe (5, Yx) 
Yielding : 


(MMe 5 Ye) — bel ( sey) -rly, | ( MhaiSy). 
(s¢y,) 





E 
diy k+l (25) 


1 1 
We applying the inequality w'v< lle + hf) with w= mares )q,., and v=m(n,,,5,)y, where 


1 
me (—=,v2r ], to the first term of the above equality, we get : 


V2 





1] 1 
(1s: (S; Meas 0S; Y,) S sl (5; y,) ‘Pnal +m°( (S,et) | ] 26) 
This yields : 
1 T 2 2 m T Ds 2 
ne (5, Y,) neal +] ——— 1 (5; Must) ly, 
m 2 
cE ees oe 


(7 y,)° 


from (23) we get : 


1 1 
dias S ea ee =a < 4 as. |’ (28) 


Therefore, we get : 


1 
dete ! - ae |’ (29) 


Next we will show that CG methods with BAH converges globally. 


3.2. Global convergence property 

Dai et al. expressed in [9] that the subsequent result had been basically established Zoutendijk and 
Wolfe. 
Lemma 1. 


Let assumptions (i) and (ii) holds. The @, is take by the Wolfe line search (4) and (5). If: 


1 
ye, oy 


20 |, 
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then 
a inf le. 21 =0 GD) 
Theorem 3.2 


Presume that the states in Assumption hold. If {d k i} and {n, a} are generated by new technique, 
then lim inf le7 | =0. 


Proof : 
From (6) and definition of B , by (19) we get : 


























ld. =|- Nea t+ Br d,| s reel +1. ld, | 
yell Irae 
dal Sapa +] oy er ss cet ls 
fasta [oer Ba, | ial 
lf erica 
Palle 
Isle 
$|Preall+ Is. (32) 
i Isc 
<Hteal/+ [vs lla + eras ll I, 
= k+1 k 
lsc lly 


= [2 + r| ell 


This relation explain to facilitate : 


ys 2( I )pzI=« (33) 
k>1 


kel d,.,|- ~\24+r 











Consequently, from Lemma 1 we have lim inf WA al = 0), which for target perform is uniformly, 
k>00 


then equivalent to lim ley | =0. 
k>00 


4. NUMERICAL RESULTS 

We tested BAH-algorithm. The test functions and their primary values are wan from [10]. 
Furthermore, Optimization problems used in many papers for example, see [11-18]. In addition to these 
functions, there are various other functions that havebeen used for testing in the folloowing research [19-25 ]. 
The numerical results are reported in Table | : “the first column and the second one represent the problem 
name and its dimension in [10], respectively. NI, NR and NF in the table denote the number of iterations, 
function evaluations and the number of restart calls, respectively”. 

All the algorithm area unit enforced in Fortran 90 ninety. All told cases, double preciseness 


arithmetic were used. The parameters in Wolfe states area unit set as ron = (0.001 and o> = 0.9. BAH- 
algorithm is efficient we see from Table 1. 
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Table 1. The Numerical Results of the FR and BAH Methods 








P. No. n FR algorithm BAH algorithm 
NI NR NF NI NR NF 
1 100 47 18 93 42 21 95 
1000 78 45 131 39 18 86 
2 100 32 15 52 22 10 41 
1000 22 10 42 23 12 44 
3 100 25 11 43 24 9 45 
1000 46 28 741 25 5 54 
4 100 32: 13 64 35 16 68 
1000 77 46 129 28 11 55 
5 100 15 6 25 17 9 29 
1000 F F F 26 20 281 
6 100 37 8 67 43 18 69 
1000 73 27 115 58 20 91 
7 100 89 32 174 72 45 163 
1000 107 40 211 88 52 226 
8 100 32 12 65 24 14 54 
1000 53 22 116 36 21 89 
9 100 9 4 18 10 6 18 
1000 12 d. 82 9 7 55 
10 100 74 21 123 93 30 140 
1000 370 88 616 341 77 567 
11 100 69 50 1202 54 37 653 
1000 98 82 1967 46 33 502 
12 100 23 11 45 21 12 40 
1000 27 11 55 17 9 41 
13 100 49 22 66 18 12 33 
1000 129 67 166 13 9 26 
14 100 122 62 156 14 9 25 
1000 130 66 166 16 10 29 
15 100 112 55 147 44 19 65 
1000 110 54 145 61 34 82 
Total 1999 933 7022 1333 585 3485 





F: The algorithm fail to converge. 

Problems numbers indicant for : “1. is the Extended Rosenbrock, 2. is the Extended Beale, 3. is the Generalized 
Tridiagonal 1, 4. is the Extended Tridiagonal 1, 5. is the Extended Three Expo Terms, 6. is the Generalized Tridiagonal 2, 7. is the 
Extended Maratos, 8. is the Extended Quadratic Penalty QP2, 9. is the ARWHEAD (CUTE), 10. is the Partial Perturbed Quadratic, 
11. is the EDENSCH (CUTE), 12. is the LIARWHD (CUTE), 13. is the DENSCHNC (CUTE), 14. is the Extended Block-Diagonal 
BD2, 15. is the Generalized quartic GQ2”. 


Can summarize our numerical results in Table 2 based on the percentage performance for all Tools 
used in these comparisons. 


Table 2. Percentage Performance of the Methods 








NI NR NF 
FR 100 % 100 % 100 % 
BAH 66.68 % 62.70 % 49.62 % 





It is clear from Table 2 that taking, over all the tools as a 100% for FR method the BAH method has 
an improvement, in about 33% NI ; 37% NR and 50% NF, these results indicate that New method is in 
general is the best . 


5. CONCLUSIONS 
A new kind of parameter in the conjugate gradient methods for large-scale unconstrained 
optimization problems is proposed. Reveal Numerical that the new method is superior in practice with 


competitive FR method. We choose the parameter (3, appropriately, to boost the performance of the 


conjugate gradient methods. 
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